feat(desktop): gated ui_screenshot — the visual representation (D3) - #487
Conversation
The desktop UI's third representation (ADR-062 D-4): pixels for the residue structure cannot answer — a rendering bug, a layout question, "why does this look wrong". It is also the most sensitive artifact the app can emit, so three rules bind every call and all three are decided in one electron-free module (uicapture.ts): - REFUSAL BY TABLE. ui_policy.ts's `capture` column decides, reading the same projected focus snapshot the agent can already see. A split refuses if EITHER pane refuses, so the vault cannot ride along in the half the user is not focused on; an undeclared surface and an unpublished screen refuse too — we decline what we cannot classify. Guests go through the same table via surfaceForPartition, so a new partition opts in by naming its row instead of inheriting one. - PER-CALL APPROVAL, ALWAYS. A `desktop_action` attention card, never a session grant. The hub raises it for relayed calls (D5); a LOCAL call never touches the hub, so the desktop raises the same kind itself and parks on it — 2 minutes, not the hub's 10, because this leg holds an MCP call open over stdio. `via: 'hub'` skips the local card, or a remote agent would be double-prompted. Signed out means no way to ask, which means refused — never captured unasked. - FAIL CLOSED. No answer in the window, a dismissed row, an unreadable resolution: all deny. The window leg re-checks the refusal after the park, because the approval was for what was on screen when the card went up. Read SCOPE, action CLASS: the tool sits in READ_TOOLS so the local kimi-code loop (read token only) can reach it — the consent that governs a capture is the card, not the spawn flag — but it audits on every leg (local browser reads do not) and hub-mirrors on every leg (hub-leg reads are ring-only). DESKTOP_ACTION_TOOL_NAMES is that distinction, in one place. The AttentionDock grows a desktop_action branch showing what was asked for — surfaces, or the fragment-stripped guest URL — with Allow once / Reject and a line saying why there is no session option. 21 new node --test cases (283 pass). The Electron halves (capturePage, the hub approval round trip) are unexercised here — no display, no Electron binary — and need the desktop/Playwright pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
28dc9d6 to
665f20d
Compare
|
Rebased the whole stack onto Post-rebase, on each branch in turn: |
|
Review of D3 (verified against head Major — timeout-dismiss is broken end-to-end (hub FK violation)
body: JSON.stringify({ resolved_by: 'desktop' }),The hub column is Fix: omit Minor
|
…estable hub leg Review fixes for #487 (D3): - The timeout-dismiss sent resolved_by:'desktop'; the hub column REFERENCES agents(id) with FK enforcement on, so the UPDATE 500'd, the catch swallowed it, and every timed-out card loitered in the director's inbox forever. The dismiss body is now {} — NULL is the human/system-dismiss convention in handleResolveAttention. - The raise/poll/dismiss round trip moves to uicapture_hub.ts, electron-free with an injectable fetch, and gains 9 node tests — one pins the FK-sensitive empty dismiss body, the rest cover approve/deny/timeout/raise-failure sequencing. - A reachable hub that refuses the card now reports its own denial message (raise_failed) instead of the signed-out one. - Comments: the visibleSurfaces split-refusal invariant (rests on every capture:'refuse' row being splitEligible:false), and why the card row's actor stays the desktop user (a user token attributing rows to arbitrary agents would be a spoofing lever). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Second review pass (agent): verified the fleet review's findings against code and fixed them on the branch — commit Major (confirmed + fixed): the timeout-dismiss body sent Also in the commit:
Verified at the new head: electron 292/292 (283+9), tsc clean both sides. 🤖 Generated with Claude Code |
…490) The stacked PRs squash-merged into their base branches (the GitHub stack cascade), so main received only D3 (#487); this merge brings the cascaded j8-d5-remote-desktop-ui head — element-resolved pointing (D4), hub-relayed desktop UI context (D5), and agent pointing (D6), each including its review-fix commit — onto main. Conflicts (main's D3-squash regions vs the branch's D5/D6-extended versions of the same code) resolved by taking the branch side, which is a strict superset; the plan's status block updated to all-shipped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2 participants
D3 of
docs/plans/desktop-ui-context-and-pointing.md— the third representation of the desktop UI entity (ADR-062 D-4): pixels for the residue structure cannot answer.Code only. The Electron halves (
capturePage, the hub approval round trip) are unexercised on the authoring machine — no display, no Electron binary — and need a desktop/Playwright pass.The three rules, all in one electron-free module
desktop/electron/src/uicapture.tsdecides all of them, sonode --testproves them without Electron.1. Refusal by table.
ui_policy.ts'scapturecolumn decides — read from the same projected focus snapshot the agent can already see, not a second source of truth. A split refuses if either pane refuses (the vault cannot ride along in the half the user is not focused on); an undeclared surface and an unpublished screen refuse too. Guests go through the same table viasurfaceForPartition, so a new partition opts in by naming its row.2. Per-call approval, always. A
desktop_actionattention card, never a session grant. §3.3 says "a hub card" and the hub raises it for relayed calls (D5) — but a local call never touches the hub, so the desktop raises the same kind itself and parks on it.via: 'hub'skips the local card, or a remote agent would be double-prompted. Signed out ⇒ no way to ask ⇒ refused, never captured unasked.3. Fail closed. No answer in the window, a dismissed row, an unreadable resolution: all deny. The window leg re-checks the refusal after the park — the approval was for what was on screen when the card went up.
Read scope, action class
ui_screenshotsits inREAD_TOOLSso the local kimi-code loop (read token only) can reach it — the consent that governs a capture is the card, not the spawn-timebrowser_bridge: trueflag. It is an action everywhere else: audited on every leg (local browser reads are not) and hub-mirrored on every leg (hub-leg reads are ring-only).DESKTOP_ACTION_TOOL_NAMESholds that distinction in one place.The park is 2 minutes, not the hub's 10: this leg holds an MCP call open over stdio, and a client that gives up mid-park would leave the user approving a call nobody is waiting for. Timeout dismisses the row and denies.
Files
uicapture.ts(new)uicapture_host.ts(new)capturePage+ the hub approval round trip; registers itself on the bridgebrowserbridge.tsUI_TOOL_NAMEScatalog gate, thecaptureUidep, audit classificationAttentionDock.tsxdesktop_actioncard branch — Allow once / Reject, and a line saying why there is no session optionVerification
npm test(electron): 283 pass, 0 fail — D3 adds 21 (uicapture.test.ts9,uiscreenshot.test.ts12)tsc --noEmit: clean, both frontend and electronnode --test src/state/*.test.ts src/ssh/*.test.ts: 376 pass (CI does not run these)lint-desktop-tokens.sh: clean at baseline 65, phantom 0 ·lint-docs.sh/lint-glossary.sh: cleanFirst of a stack: D3 → D4 → D5 → D6.
🤖 Generated with Claude Code